Structured Semantic Transfer for Multi-Label Recognition with Partial Labels

Multi-label image recognition is a fundamental yet practical task because real-world images inherently possess multiple semantic labels. However, it difficult to collect large-scale multi-label annotations due the complexity of both input and output label spaces. To reduce annotation cost, we propose structured transfer (SST) framework that enables training models with partial labels, i.e., merely some labels are known while other missing (also called unknown labels) per image. The consists two complementary modules explore within-image cross-image correlations knowledge generate pseudo for Specifically, an intra-image module learns image-specific co-occurrence matrix maps complement based on this matrix. Meanwhile, category-specific feature similarities helps high similarities. Finally, generated used train models. Extensive experiments Microsoft COCO, Visual Genome Pascal VOC datasets show proposed SST obtains superior performance over current state-of-the-art algorithms. Codes available at https://github.com/HCPLab-SYSU/HCP-MLR-PL.